MEMORANDUM October 31, 2016 
TO: Caleen Allen 


FROM: Carla Stevens 
Assistant Superintendent of Research & Accountability 


SUBJECT: STACEY AND BO PORTER SELF FOUNDATION AFTERSCHOOL PROGRAM: 
ACADEMIC AND SOCIAL IMPACTS ON KEY, REVERE, AND MEYERLAND MIDDLE 
SCHOOLS, 2015-2016 


CONTACT: Carla Stevens, (713) 556-6700 


During the 2015-2016 academic year, the Stacey and Bo Porter SELF (Sports, Education, Life-Skills, 
and Faith) Foundation involved 255 at-risk students at Key, Revere, and Meyerland middle schools in 
the afterschool program. The program offered academic support, life skills, sports, and spiritual 
enrichment through tutorials, guest speakers, and inspirational messages on self-esteem and 
character-building. The DOORS program (Discovery Opportunities that Offer Real Success) was 
implemented to assist eighth-grade students transition to high school. Program impact was measured 
using the State of Texas Assessment of Academic Readiness (STAAR) reading and math 
performance, attendance, and disciplinary actions of SELF students compared to non-program 
students enrolled at SELF schools. 


Key Findings: 

e SELF students achieved higher increases in STAAR mean scale scores from fifth to sixth grade 
in math, sixth to seventh grade in reading and math, and seventh to eighth grade in math relative 
to comparison-group students. 


e Propensity score matching using STAAR of eighth-grade SELF students who participated in the 
DOORS program, revealed an estimated increase of nearly 10 scale score points in reading and 
math due to SELF program participation. 


e Disciplinary outcomes were better for SELF students compared to non-SELF students relative to 
the percentage of students with “no” in-school and out-of-school suspensions, alternative 
placements, or other disciplinary referrals retrospectively in 2016 from 2015. 


e The mean numbers of excused, unexcused, and total absences were lower for SELF students 
relative to the non-SELF comparison group. 


e The vast majority of students indicated that they were benefitting from all program components, 
with the largest majority specifying physical activities (93%) and tutorials (90%). The Foundation 
should consider expanding these components, while building character traits that support positive 
self-esteem, community service, safety, and preparation for future education and careers. 


Should you have any questions or require any further information, please contact me in the 
Department of Research and Accountability, at 713-556-6700. 


CJS 
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cc: Grenita Lathan 
Rose Adams 
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Stacey and Bo Porter SELF Foundation Afterschool Program: Academic and Social 
Impacts on Key, Revere, and Meyerland Middle Schools, 2015-2016 


By Venita Holmes, Dr.P.H. 


The SELF Foundation afterschool program involved 255 students at Key, Revere, and Meyerland (formerly 
Johnston) middle schools in tutorials and enrichment activities to support their academic, social, and emotional 
development. All students at SELF schools were allowed to participate in a character-building essay contest for 
prizes. Program impact was measured using STAAR reading and math performance, attendance, and disciplinary 
actions of SELF students compared to non-program students enrolled at SELF schools. Both groups showed gains 
in reading and math from 2015 to 2016; however, SELF students achieved higher increases in mean scale scores 
from 5" to 6" grade in math, 6" to 7" grade in reading and math, and 7" to 8" grade in math. Propensity score 
matching using STAAR of 8'"-grade SELF students who participated in the DOORS program, revealed an 
estimated increase of nearly 10 scale score points in reading and math due to SELF program participation. 
Disciplinary outcomes were better for SELF students compared to non-SELF students relative to the percentage of 
students with “no” in-school and out-of-school suspensions, alternative placements, or other disciplinary referrals 
retrospectively in 2016 from 2015. The mean numbers of excused, unexcused, and total absences were lower for 
SELF students relative to the non-SELF comparison group. The vast majority of students indicated that they were 
benefitting from all program components, with the largest majority specifying physical activities (93%) and 
tutorials (90%). The Foundation should consider expanding these components, while building character traits that 
support positive self-esteem, community service, safety, and preparation for future education and careers. 


Background 


The Stacey and Bo Porter SELF Foundation 
afterschool program supports youth development by 
offering activities to improve lives through Sports, 
Education, Life Skills, and Faith. The program 
emphasizes youth fitness, education, social well-being 
and spiritual enrichment as essential building blocks to 
nurture the whole youth athletically, academically, 
socially, and spiritually. SELF thrives to enhance the 
overall success of communities. The program was 
initiated in the Houston Independent School District 
(HISD) during the 2013-2014 academic year as a pilot 
program at Key Middle School and expanded to Revere 
and Welch middle schools during the 2014-2015. Key 
students are in their third year, while Revere students 
are in their second year of the program. Johnston (now 
referred to as Meyerland) Middle School was added in 
2015-2016, and Welch is no longer participating in the 
program. 


Figure 1 reflects the number of SELF students over 
the past two years. All students at SELF schools were 
given the opportunity to participate in the program; 
however, only students whose parents provided signed 
consent were SELF students. Key students were no 
longer provided transportation during the 2015-2016 
academic year, resulting in a decline in participation. 


2014-2015 2015-2016 
= Meyerland 104 
m Key 127 83 
= Revere 75 78 
m= Welch 85 
= All SELF Students 287 255 


Figure 1: SELF students by School, Past Two Years 


SELF Program Key Components 

e Afterschool Program provided a wide array of 
academic and enrichment activities to SELF 
students, including tutorials, life skills, character 
building, sports, spiritual enrichment, and field trips. 
Guest speakers and inspirational messages by Bo 
and Stacey Porter were central components of the 
program. Enrichment activities, including club 
baseball, flag football, soccer, and lacrosse, were 
offered to all students in sixth, seventh, and eighth 
grades. 


e Character-building Essay Contest required effective 
communication, writing, and networking skills for 
students to earn prizes. All students at SELF 
Foundation schools had the opportunity to 
participate in the essay contest. 


e Boys to Men Program addressed social factors that 
contribute to outcomes associated with minority 
males. Focus on academic achievement, motivation, 
enhancing potential, and providing opportunities 
through mentorship and resources supported growth 
and mastery in all areas of life. 


e Tom Wadley First Pitch Baseball Tournament was 
held on February 5, 2016 at Johnston Middle School 
and February 6, 2016 at Key Middle School on the 
Fondren Field to foster teamwork and character- 
building. These events were supported by the 
Fondren Family Foundation and Academy Sports 
and Outdoors. 


e DOORS program (Discovery Opportunities that 
Offer Real Success) was implemented as a 
supplement to the afterschool program to help 
eighth-grade students transition to high school. 
Topics of discussion were conflict resolution, test- 
taking strategies, high-school clubs and activities, 
getting to know your counselor, and how to 
calculate your GPA. Activities were facilitated by 
certified HISD teachers. 


Site coordinators helped to coordinate SELF program 
activities at their school, while a program director, 
employed by the SELF foundation, provided oversight 
of the full program. SELF afterschool program students 
were provided dinner at the end of the day at Key and 
Revere middle schools. 


Review of the Literature 


There are contrasting views regarding the impact of 
afterschool programs on_ students’ growth and 
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development. Many educators believe that afterschool 
programs are vital to ensure that children are safe, while 
providing opportunities for them to engage in 
academically and_ socially-enriching activities that 
support parents during out-of-school hours (U.S. 
Department of Justice, 2001; Afterschool Alliance, 
2013). A meta-analysis of 68 afterschool studies found 
that students participating in high-quality afterschool 
programs went to school more, behaved better, received 
better grades and performed better on tests compared to 
non-participants (Durlak, Weissberg, & Pachan, 2010). 
Research on nearly 3,000 low-income students at 35 
high-quality afterschool programs across the United 
States found that students who regularly attended 
afterschool programs, compared to their routinely 
unsupervised peers, made significant gains in their 
standardized math test scores; experienced reductions in 
teacher-reported misconduct, and reduced drug and 
alcohol use over two years (Belsky, et. al., 2007). 
Further, after controlling for baseline obesity, poverty, 
race and ethnicity, the prevalence of obesity was 
significantly lower for afterschool program participants 
compared to non-participants (Mahoney, Lord, and 
Carryl, 2005). 

The U.S. Department of Education (2014) funds 
afterschool programs through 21st Century Community 
Learning Centers to support education and enrichment, 
specifically for students who attend high-poverty and 
low-performing schools. A report released in 2004 
found that academic test scores of student participants 
were no better than scores of students not involved in 
the programs and, in some cases, behavior appeared to 
worsen (Ed Week, 2004). However, Shernoff (2010) 
found that the “quality of experiences in after-school 
programs may be a more important factor than quantity 
of experiences (i.e., dosage) in predicting positive 
academic outcomes” (p. 325). 

While trends have varied relative to the impact of 
afterschool programs on students’ academic, social, and 
emotional development, an in-depth examination of 
specific program activities among targeted student 
populations is needed to clearly understand which 
programs work, for whom, and under what 
circumstances. To that end, this report is designed to 
explore factors that impact student’s academic 
performance, school attendance, and discipline. The 
report also offers insight concerning which components 
students considered beneficial toward enhancing their 
social and emotional growth and development, as well 
as their perceptions relative to safety, education, and 
developmental assets. 


Methods 


A mixed-method study was conducted, using both 
qualitative and qualitative measures. 
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Table 1: Total Students by SELF School, SELF and Non-SELF Student Samples, 2015-2016 


Meyerland MS 


Key MS 


Revere MS SELF 


Student Sample 


Non-SELF 
Student Sample 


Asian 2 0 3 4 2 121 4 
African American 42 61 29 114 45 1,205 35 
Hispanic 48 37 58 121 48 1,709 50 
White 6 0 6 10 4 334 10 
Two or More Races 2 0 3 4 Z 25 1 
Indian/Pacific Is. i 0 2 1 17 <1 
————— es a aes ee ae ee Eee 
Male 69 49 57 151 59 1657 48 
Female 31 51 43 104 41 1,754 51 
Other Characteristics | PP 
Economic Disadvantaged 79 66 79 191 75 2,562 75 
At Risk 53 74 56 154 60 2,038 60 
Gifted/Talented (G/T) 21 0 3 23 9 585 17 
Limited English Proficient (LEP) 22 19 31 60 24 913 27 
Special Education 6 7 8 18 7 322 9 


Source: PEIMS, 2015-2016 


SELF Student Sample 

Site coordinators and school administrators provided 
a list of SELF students to the HISD Research and 
Accountability Department. These lists were combined 
to form the SELF student sample. A profile of the 255 
students can be found in Table 1. A decline in SELF 
participation at Key may be related to students not 
being provided transportation in 2015-2016. 


SELF vs. Non-SELF Student Comparison Group 

All students at SELF schools who were not in the 
afterschool program comprised the non-SELF student 
comparison group for the analyses. Both SELF and 
non-SELF students were predominately economically 
disadvantaged and at risk. A higher percentage of non- 
SELF students were gifted/talented compared to the 
SELF student sample. However, lower percentages of 
limited English proficient (LEP) and special education 
students were SELF students compared to non-SELF 
students. 


Measures and Variables 

Key variables assessed in the quantitative analyses 
were academic achievement, attendance, and 
disciplinary action rates for the 2014-2015 (pretest) and 
2015-2016 (posttest) academic years. Specifically, 
academic achievement of SELF and non-SELF students 
was measured using students’ 2015 and 2016 reading 
and math scale scores on the State of Texas 
Assessments of Academic Readiness (STAAR) to 
compare change in performance between the groups 
over the two-year period. 


The STAAR is aligned with the state curriculum 
standards and the Texas Essential Knowledge and 
Skills (TEKS). STAAR standards are designed to 
prepare students for postsecondary education and to 
ensure that they are competitive with other students 
both nationally and internationally (TEA, 2010). 

Attendance data included excused and unexcused 
absences for the 2014-2015 and the 2015-2016 
academic years. Attendance data were captured from 
the data warehouse Cognos Chancery package, August 
9, 2016. 

Student discipline was based on the number and 
percent of “no” in-school and__ out-of-school 
suspensions, alternative placements, expulsions, and 
“other” disciplinary actions that students incurred 
during the 2014-2015 and the 2015-2016 academic 
years. A paired sample was created with students who 
had disciplinary data during the 2015-2016 school year 
(posttest) and disciplinary data in 2014—2015 (pretest). 
Disciplinary data were extracted from Cognos, August 
9, 2016 for SELF and non-SELF students. The groups 
were compared to determine whether the prevalence of 
disciplinary actions changed over time. 

Qualitative analysis was also conducted based on a 
paper-and-pencil survey that was administered to SELF 
students in May 2016. The survey measured students’ 
perceptions relative to: (1) safety, education, and 
developmental assets (17 items) (SEARCH Institute, 
2014); (2) benefits of program components (5 items); 
and (3) social and emotional interests and needs (19 
items). Finally, SELF students were asked to express 
their feelings about the program in an open-ended 
question format. A total of 86 SELF students completed 
the survey, yielding a 30% survey participation rate. 


What was the impact of the SELF program on the 


student’s academic achievement? 


The impact of the SELF program was measured 
using the 2015 (pretest) and the 2016 (posttest) STAAR 
reading and math subtest results (first test 
administration). The mean scale scores of SELF 
students were compared with all other students at SELF 
schools based on paired t-test analyses. Analyses were 
conducted for SELF and non-SELF student groups with 
both 2015 and 2016 STAAR reading and math data. 

Figure 2 presents the STAAR reading and math 
results of SELF students who participated in the 
program as 6" graders in 2015-2016 (posttest) and who 
were 5" graders in 2015 (pretest). Non-SELF students 
with STAAR data in comparable years are also 
presented in Figure 2. Both groups showed gains in 
reading and math. SELF students’ reading scores 
increased by 15 points; whereas, non-SELF students 
reading scores increased by 46 points. On the math test, 
SELF student scores increased by 66 points, while non- 
SELF students’ math scores increased by 56 points. 
Thus, SELF students demonstrated greater gains in 
math compared to the non-SELF student group. 

Figure 3 depicts the STAAR reading and math 
results of SELF students who participated in the 
program as 7"" graders in 2015-2016 (posttest) and who 
were 6'" graders in 2015 (pretest). Non-SELF students 
with STAAR data in comparable years are also depicted 
in Figure 3. 

It is evident that 7 grade SELF and non-SELF 
student groups showed gains in reading and math 
(Figure 3). SELF students’ reading scores increased by 
76 points; whereas, non-SELF students reading scores 
increased by 64 points. On the math test, SELF student 
scores increased by 53 points, while non-SELF 
students’ math scores increased by 48 points. 


Grade 5 Grade 6 
(pretest -2015) | (posttest -2016) 
Reading Math 
SELF 1539 1554 1566 1632 
Non-SELF 1545 1591 1600 1656 


Grade 5 Grade 6 
(pretest -2015) | (posttest -2016) 


|SELF ®&Non-SELF 


Figure 2: SELF students vs. non-SELF students, paired sample, 
reading and math STAAR, spring 2015 (pretest) vs. 2016 
(posttest), average scale score 
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1700 
1650 
1600 
1550 


1500 


1450 
Grade 6 Grade 7 Grade 6 Grade 7 


(pretest -2015) (posttest -2016) (pretest -2015) (posttest -2016) 
Reading Math 

SELF 1561 1637 1617 1670 

Non-SELF 1624 1688 1613 1661 


™SELF ™Non-SELF 


Figure 3: 6" grade SELF vs. non-SELF students, paired reading 
and math STAAR, spring 2015 (pretest) vs. 2016 (posttest), 
average scale score 


Consequently, SELF students demonstrated greater 
gains in reading and math compared to the non-SELF 
student group (Figure 3). 

Figure 4 compares the STAAR reading and math 
results of SELF students who participated in the 
program as 8" graders in 2015-2016 (posttest) and who 
were 7'" graders in 2015 (pretest). Non-SELF students 
with STAAR data in both years are also shown in 
Figure 4. 

It can be seen in Figure 4 that 3” grade SELF and 
non-SELF student groups demonstrated increases in 
their mean reading scale scores. SELF students’ mean 
reading scores increased by 53 points compared to non- 
SELF students, whose mean reading scores increased 
by 55 points. On the math test, SELF student scores 
increased by 79 points compared to non-SELF students 
whose math scores increased by 71 points. SELF 
students showed greater gains in math relative to the 
non-SELF student group. 


1720 
1680 


1640 


1600 
” a | 
1520 


Grade 7 Grade 8 Grade 7 Grade 8 
(pretest -2015) (posttest -2016) (pretest -2015) (posttest -2016) 
Reading Math 
SELF 1598 1651 1581 1660 
Non-SELF 1623 1678 1599 1670 


= SELF ™Non-SELF 


Figure 4: 8" grade SELF vs. non-SELF students, reading and math 
STAAR, spring 2015 (pretest) vs. 2016 (posttest), average scale 
score 


What was the treatment effect of the SELF 


program on the student’s academic achievement? 


The treatment effect of the program was determined 
using the results of SELF students who were 8" graders 
during the 2015-2016 academic year, considering that 
these students were engaged in an additional program, 
DOORS (Discovery Opportunities that Offer Real 
Success), to help them transition from middle school to 
high school. 

Propensity score, nearest neighbor matching created 
comparable 8'"-grade student groups using students’ 
2016 STAAR reading and math scale scores as the 
outcome measures, controlling for at-risk status and 
2015 reading and math scales scores. STATA software 
was used for matching SELF students with non-SELF 
students. Propensity score matching yielded 75 SELF 
students and 905 non-SELF students in the reading 
performance analyses. In addition, 62 SELF students 
and 748 non-SELF students were identified in the math 
propensity score analyses. 

Figure 5 shows that before propensity score 
matching, SELF students had a lower STAAR mean 
reading scale score than non-SELF students (1648 vs. 
1671). However, after controlling for at-risk status and 
2015 reading scale scores, SELF students had a higher 
STAAR mean reading scale score than non-SELF 
students (1648 vs. 1537). STATA propensity matching 
estimated the average treatment effect (teffects) to be a 
9.7 scale score increase on the STAAR reading test for 
8'"-grade students due to participation in SELF. 

Relative to math, Figure 6 shows that before 
propensity score matching, SELF students had a lower 
STAAR mean math scale score than non-SELF students 
(1652 vs. 1659). After controlling for at-risk status and 
2015 math scale scores, SELF students had a higher 
STAAR mean math scale score than non-SELF students 
(1651 vs. 1637). STATA propensity matching estimated 
the average treatment effect (teffects) in math as 9.8 
points due to participation in the program. 


1680 


1659 


1660 


1651 


1640 


1620 


Unmatched-Before Program Matched-A fter Program Effect 


™SELF @Non-SELF 


Figure 5: SELF program effects on gt graders, propensity 
score matching, controlling for 2015 STAAR reading scores 
and at risk status, with 2016 reading scores as outcome 
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Po gn oe pao GaN Een wanes ge eed ei Af anccks See et eee 
Figure 6: SELF program effects on 8" graders, propensity score 


matching, controlling for 2015 STAAR math scores and at risk 
status, with 2016 math scores as outcome 


T-effects revealed an estimated 9.8 scale score 
increase on the STAAR reading test for 8"-grade 
students due to participation in the SELF program. 


How did SELF impact student’s school disciplinary 


actions? 


In order to determine the impact of SELF on 
students’ behavior, disciplinary referrals were measured 
for a paired sample SELF and non-SELF students at 
the same schools for the current school year as a 
posttest measure (2015-2016) and the previous school 


year aS a pretest measure (2014-2015) to detect 


changes among the samples. Appendix A and Figures 
7 and 8 display the distribution of “no” disciplinary 
actions among the paired samples of SELF and non- 
SELF students over the two-year period. Figure 7 
shows that higher percentages of SELF students 
compared to non-SELF students had “no” in-school 
suspensions during the 2014-2015 school year; 
whereas, higher percentages of non-SELF students had 
“no” occurrences of out-of-school suspensions and 
“other” disciplinary actions. Examples of “other” 
disciplinary actions were student or parent conference, 
detention, behavior/conduct contract, and referral to a 
counseling agency. 


8 


99.7 100.0 
92.6 92.3 88.5 


| 74.5 
57.7 
41.6 
19.2 26.5 
+t : fi : 


‘ T 1 
In-School Out-of-School 
Suspensions Suspensions 


Percent "No" Actions 
N L a co 
o o o o 


o 


Alternative 
Placements 


Expulsions Other 


| SELF Students = Non-SELF 


Figure 7: Paired analyses, percent of students with “No” 
disciplinary actions SELF and Non-SELF student samples at same 
schools, 2014-2015 (28 SELF students, 310 non-SELF students) 
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Figure 8: Percent of students with “No” disciplinary actions SELF 
and non-SELF student samples at same schools, 2015-2016 (80 
SELF Students, 697 Non-SELF Students) 


During the 2015-2016 school year, SELF students 
maintained higher rates of “no” in-school suspensions, 
but had higher rates of “no” out-of school suspensions, 
and “no” alternative placements compared to non-SELF 
students during the 2015-2016 school year (Figure 8). 
Both groups had no expulsions in 2015-2016. 

Notably, over the two-year period, the percentage of 
students with “no” in-school suspensions dropped for 
SELF (16.3 points) and non-SELF students (8.9 points), 
while the percentage of students with “no” out-of- 
school suspensions increased for both student groups 
(27.1 points vs. 10.9 points). 


How did the rate of absenteeism for SELF students 


compare to students at SELF schools? 


The mean number of excused, unexcused, and total 
absences is presented for SELF students and a non- 
SELF comparison-student group in Figure 9. 
Additional attendance data can be found in Appendix 
B. SELF students had a lower mean number of excused, 
unexcused, and total absences than non-SELF students. 


Mean 
COorR N WH UU DN OD 


Unexcused Absences Total Absences 


Excused Absences 


M™SELF Students ® Comparison Group 


Figure 9: Mean number of days absent for SELF and non-SELF 
student samples at SELF schools, 2015-2016 (92 SELF Students, 
1,153 non-SELF students) 
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What were SELF student’s perceptions regarding 


the afterschool program? 


SELF students were asked to indicate whether or not 
the program benefitted them in school or in their 
personal life “now.” Results from the 82 students who 
completed the survey are depicted in Figure 10. The 
majority of surveyed students replied that they currently 
benefit from each of the program components (i.e., 
Physical Activities, Character Building, Guest 
Speakers, Tutorials, and the Quote of the Day). The 
highest percentage of students expressed benefits from 
Physical Activities (94%), followed by Tutorials (78%). 
Surveyed students were less likely to reveal current 
benefits from the Quote of the Day (66%). This pattern 
is consistent with the previous evaluation. 
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Figure 10: Percentage of SELF students who indicated whether 
or not they were benefitting “now” from SELF program 
activities, 2015-2016 


Table 2 shows the percentage of students who 
responded whether or not the program would benefit 
them in the “future” at school or in their personal life. 
The highest percentage of students indicated that they 
would benefit from topics related to “Hard Work” 
(94%), followed by “Manners and Respect” (89%). In 
contrast, the lowest percentage of students noted that 
they would benefit in the future from topics related to 
“Financial Literacy” (66%), followed by “Financial 
Budgeting” (68%). 

Table 3 presents reflections of SELF students 
regarding their behavior or feelings since participating 
in the program. Key findings revealed that 74% of 
survey respondents indicated that they “almost always” 
feel safe at home compared to 59% who indicated 
feeling safe in their neighborhoods, and 47% who 
indicated feeling safe at school. The vast majority of 
students replied that they “almost always” plan on 
completing their high school education (91%) and plan 
on attending college after high school (90%). Only 29% 


Table 2: Percent of SELF students who indicated they 


would benefit in the “future” from program-related 
topics, 2015-2016 


Topic % Yes 
2015-2016 
(n=82) 

Hard work 94 
Manners and Respect 89 
Perseverance 88 
Preparation 87 
Trustworthiness 87 
Attitude 86 
Etiquette 86 
Goal Setting 85 
Commitment 85 
Appropriate Use of Social Media 85 
Accountability 83 
Punctuality 83 
Time Management 81 
Proper Language & 81 
Communication 

Networking 79 
Hand Shaking and Eye Contact 73 
Financial Budgeting 68 
Financial Literacy 66 


of students indicated that they “almost always” serve in 
their community. Forty-three percent of students 
revealed that they are usually happy with the decisions 
that they make. 

Students were asked to provide their overall feelings 
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about the program. In general, the comments were 
positive. Some comments were: 


“T think the afterschool program is a great way to 
help kids.” 

“T think the Bo Porter Self Foundation Program is 
great.” 

“T play baseball and feel that it's helpful.” 

“T feel safe in the program and it helps me do or 
get help with problems I have and do my 
homework.” 

“Very fun and makes me get in shape.” 


Students also mentioned receiving help with 
homework assignments, playing soccer, and content 
area courses, and being able to eat dinner. 


Discussion 


Research has shown that youth participating in 
quality afterschool programs gain social, emotional, and 
behavioral benefits (U.S. Department of Education, 
2003, Shernoff, 2010). The Stacey and Bo Porter SELF 
Foundation implemented a pilot afterschool program at 
Key Middle School during the 2013-2014 academic 
year. The program expanded in 2014-2015 to include 
students at Revere and Welch middle schools. During 
the 2015-2016 academic year, 255 students participated 
in the afterschool program at Key, Revere, and 
Meyerland middle schools. A total of 3,411 students 
were exposed to various components of SELF, 
including Boys to Men, Character-building Essay 
Contest, Tom Wadley First Pitch Baseball Tournament, 
DOORS, and guest speakers facilitated by Stacey and 
Bo Porter. 

SELF 


students were provided mentoring and 


Table 3: SELF students’ reflections on behavior or feelings since participating in the SELF program, 2015-2016 


Almost Always _ Sometimes Never 

% % % 
I am good at planning ahead and making decisions. 56 43 1 
I try to do well in school. 82 19 0 
I help make decisions in my home. 46 49 5 
I can accept and take personal responsibility. 65 32 4 
I feel safe at school. 47 41 12 
I feel good about myself. 73 26 il 
I am optimistic about my future. 65 34 1 
I feel safe at home. 74 23 4 
I serve in my community. 29 48 23 
I am usually happy with the decisions that I make. 43 55 3 
I plan on completing my high school education. 91 9 0 
I plan on attending college after high school. 90 9 il 
I feel safe in my neighborhood. 59 34 7 
My parents and teachers expect me to do my best at school and in other 93 6 1 
activities. 
I am proud of who I am. 83 13 4 
I am comfortable around people of different races/ethnicities. 53 40 6 
I feel that I have some influence over things that happen in my life. 56 39 5 


coaching opportunities along with lectures on topics 
that supported strengthening developmental assets. 
Participation in sports activities, including competitive 
baseball and lacrosse, soccer, and flag football, was a 
fundamental component of the program. Tutoring was 
available to students who needed academic assistance. 

This report explored the impact of the SELF program 
by analyzing the 2016 reading and math STAAR 
performance of SELF students who participated in the 
afterschool program compared to other students at 
SELF schools. Discipline outcomes were assessed 
based on a paired sample of afterschool program SELF 
students and non-SELF students over a two-year period. 
Attendance was assessed by measuring the attendance 
rates of SELF and non-SELF students relative to 
excused, unexcused, and total absences during the 
2015-2016 school year. 

Students who participated in the SELF afterschool 
program and non-SELF student groups showed gains 
on STAAR reading and math tests. However, SELF 
program students achieved higher mean scale scores in 
math from fifth to sixth grade, in reading and math 
from sixth to seventh grade, and in math from seventh 
to eighth grade compared to the non-SELF student 
group. Further, propensity score matching of eighth 
grade students who participated in the DOORS program 
reflected an estimated gain of nearly 10 scale score 
points in reading and math due to participation in the 
program. 

Disciplinary outcomes were better for SELF students 
in relation to non-SELF students when considering the 
percentage of students with “no” in-school and out-of- 
school suspensions, alternative placements, or other 
disciplinary referrals retrospectively from 2016 to 2015. 
The mean numbers of excused, unexcused, and total 
absences were lower for SELF students compared to the 
non-SELF comparison group based on 2015-2016 
attendance data. Moreover, the majority of SELF 
students indicated that they were benefitting from all 
program components, particularly, physical activities 
and tutorials. Continued focus on interventions that 
support improvements in these areas is recommended. 

There are several limitations to this evaluation. 
Student perceptions were self-reported. However, Pace 
(1985) maintains that the quality of questionnaire 
answers (reliability, validity, credibility) depends 
primarily on the quality of the questions. The 
questionnaire included items that were developed based 
on the SEARCH Institute’s (2014) developmental 
assets. In spite of the methodological challenges, the 
evaluation revealed that the Stacey and Bo SELF 
Foundation program exposed students to a variety of 
activities that they may not have encountered without 
the program. These activities were designed to promote 
growth and personal development in youth and may 
have long-term social and academic benefits as students 
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continue their education and pursue careers. Program 
administrators may consider expanding activities that 
help students manage their behavior and improve 
attendance. Issues, such as safety and serving in the 
community may be areas for enhancement. Additional 
program components could focus on building students’ 
self-esteem and confidence to plan and make decisions 
that support positive education and career trajectories. 

Future evaluations should continue to monitor the 
academic achievement, disciplinary actions, and 
attendance of SELF Foundation students to determine 
the impact of the program on the targeted population. 
Longitudinal tracking of SELF student outcomes 
beyond middle-school may provide additional 
information relative to the long-term impact of the 
program on students’ social functioning and academic 
performance. 
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Appendix A 


Disciplinary Data Analysis 


2015-2016 | 


Group “No” Disciplinary Actions Frequency Percent 

Non-SELF Comparison Students Alternative Placement 636 91.2 
Expulsions 0 0.0 
In-School Suspensions 228 32.7 
Other Disciplinary Actions’ 525 75.3 
Out-of-School Suspensions 261 37.4 
Total 697 100.0 

SELF Students Alternative Placement 77 96.3 
Expulsions 0 0.0 
In-School Suspensions 33 41.3 
Other Disciplinary Actions' 62 77.5 
Out-of-School Suspensions 37 46.3 
Total 80 100.0 


Note: Examples of Other Disciplinary Actions include student or parent conference, detention, behavior/conduct contract, and 
referral to a counseling agency. 


“No” Disciplinary Actions 


Group Frequency Percent 


Non-SELF Comparison Students Alternative Placement 287 92.6 
Expulsions 309 99.7 

In-School Suspensions 129 41.6 

231 74.5 


Other Disciplinary Actions' 


Out-of-School Suspensions 82 


Total 


SELF Students Alternative Placement 
Expulsions 


In-School Suspensions 


Other Disciplinary Actions’ 


Out-of-School Suspensions 


Total 
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Appendix B 
Attendance Data for SELF and Non-SELF Student Groups, with 
both 2014-2015 and 2015-2016 Absences 


SELF status group N | Mean Std. Deviation 


2016 Excused Absences SELF 92 1.47 2.166 
Non-SELF 1153 2.79 4.480 
2016 Unexcused Absences SELF 92 5.77 6.438 
Non-SELF 1153 6.24 7.979 
2015 Excused Absences SELF 92 1.86 2.625 
Non-SELF 1153 2.47 3.579 
2015 Unexcused Absences SELF 92 3.68 3.863 
Non-SELF 1153 4.50 5.599 


95% Confidence 


Interval of the 


Sig. (2- Mean Difference 

Independent t test, SELF and Non- 

SELF same schools revi (ae)) Difference Lower Upper 
2016 Excused -2.796 1243 .005 -1.318 -2.244|  -.393 
2016 Unexcused -.553 1243 980 -.472 -2.146| 1.202 
2015 Excused -1.604 1243 109 -.611 -1.359 .136 
2015 Unexcused -1.382 1243 .167 -.817 -1.976 342 
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